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ABSTRACT 


An experiment is described in which optical proximity sensors were 
used in a feedback loop to automatically position a manipulator hand for 
grasping. The experiment was a simplified one, involving two dimensional 
motion only. Two proximity sensors were mounted on the hand, and control 
signals derived from their outputs were used to drive the hand vertically and 
horizontally. The sensors employ a pulsed gallium arsenide light -emitting 
diode together with a silicon detector. They indicate, without contact, the 
approximate distance between the manipulator hand and object in the range 
from 5 to 1 2 cm. Positioning within approximately ±5 mm was observed. 
Extension of the technique to general three-dimensional control is briefly 
discussed. 
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I. INTRODUCTION 


The purpose of this report is to describe an experiment in which 
proximity sensors were used in a direct feedback loop to guide a manipulator 
into position for grasping. The experiment was a simplified one, involving 
motion in two dimensions only, but it illustrates how similar techniques 
could be extended in the future to facilitate teleoperator or robot operations. 

The manipulator technology itself is well developed and has had many 
routine applications, especially in nuclear hot-shops, and underwater. The 
related literature is extensive, but the current status of manipulator and 
teleoperator development can be reviewed in Refs. 1 through 4. Generally, 
manipulation has been done using visual contact, either through a window or 
by means of TV, but without additional effector -mounted sensors. Some 
work with tactile sensing techniques has been described by Hill and 
co-workers (Ref. 5) and also by Goto (Ref. 6) and others (Ref, 7) in Japan. 

A hand-mounted optical sensor was also described by Stanford Research 
Institute (Ref. 8). However, there has been no previous demonstration of 
control by hand-mounted optical proximity sensors, which function without 
requiring actual contact between manipulator and object. 

Although almost anything a human can do can also be done remotely by 
manipulator, the time required to do a given task remotely is typically one 
to two orders of magnitude longer than required to do the same operation 
manually. We believe that proximity sensing can simulate the missing sense 
of feel, and if the sensory information can be properly coupled into the man- 
machine system, significant increases in productivity will be achieved. The 
present experiment demonstrates an initial step toward this goal. 

Specifically, a pair of optical proximity sensors were used on an 
experimental manipulator to guide the effector to a predetermined vertical 
distance from an object, under closed loop control. In addition, lateral 
motion was commanded by the same two sensors to center the effector above 
the highest point of the test object. 

The following sections describe the sensors themselves, and the con- 
figuration of the sensor -equipped manipulator as used for this experiment. 
The results, which at this point are qualitative in nature, are described. 
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Finally, problems to be expected in extending the demonstrated techniques 
into the three dimensional world are discussed. 

II. THE SENSORS 

Our proximity sensors are optical devices, based on a gallium 
arsenide LED infrared light source and silicon detector. They have been 
described in an earlier report (Ref. 9). They can detect the presence of a 
surface within a predetermined sensitive volume, essentially by triangula- 
tion, using light reflected from the surface. In addition, they indicate the 
approximate distance to the surface through the magnitude of the sensor 
output. The distance scale over which an output is generated can be altered 
by the design of a replaceable external lens. The overall size of the sensor 
head lends itself to being mounted directly on the effector, even on the 
fingers themselves. 

The two identical sensors used in this experiment were modified 
somewhat from the configuration described earlier, to satisfy the require- 
ments of this experiment. The external lens was made plano-convex, with 
a spherical surface, and had a focal length of 9 cm. It was fabricated from 
lucite, as shown in Fig. 1, with an integral dove tail for quick replacement. 
This lens replaces the prism used earlier. In the present configuration the 
small internal lenses can be permanently focused at infinity, and need not 
be refocused if the focal length of the outer lens is changed. 

To lengthen the sensitive region, thus obtaining a suitable range of 
roughly proportional response, the narrow slits were removed from the 
light source and detector. Each was replaced with a single knife edge, 
oriented as shown in Fig. 2. The knife edges are imaged in the working 
space in such a way that they tend to sharply cut off response beyond the 
focal point. Light reflected from the object is seen by the detector if the 
images of source and detector indicated in Fig. 2 overlap, which occurs if 
the surface is closer than the focal distance shown. A continuously increas- 
ing output is observed as the object is moved toward the sensor from the 
focal point, where the image of the two knife edges coincide, because 
increasing areas of the light source and detector images overlap. 
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The observed output voltage as a function of distance is shown in 
Fig, 3. The output is sufficiently well-behaved in the range from 6 cm to 
12 cm to be useful for control. An approach closer than the location of the 
peak output at 5 cm must be avoided because of possible instability. 

HI. THE EXPERIMENT 

The two proximity sensors were mounted on the effector (hand) of an 
experimental manipulator (Ref. 3, p. 74) as shown in Fig. 4. The sensitive 
volumes were placed about 3 cm each side of the central plane of the finger 
action, and just below the fingertips. The output of a sensor is thus a 
measure of the vertical (Z in Fig. 4) distance to the work surface, along the 
axis of the sensor. Increasing output indicates closer approach, as seen 
in Fig. 3. 

The manipulator was manually operated in a rate mode with individual 
controls for each joint. For this experiment, the appropriate manual inputs 
were simply switched out and replaced with sensor signals, A more effec- 
tive way of utilizing the sensor signals in a real application is suggested in 
the following discussion. Vertical motion was controlled by a signal 
obtained by comparing the sum of the two sensor outputs with an adjustable 
reference indicated in Fig. 3 by V . The control loop was arranged to move 
the hand such that the sensor output was driven toward the reference level, 

V . Thus, the hand is forced to a null position at a predetermined vertical 
distance from the surface under it. 

Since the sensors were identical, the difference in their outputs is a 
measure of height difference and thus of the surface slope in the x direction. 
This difference was used directly to drive the hand in the x direction. The 
polarity was arranged to drive the hand uphill, that is toward a local 
maximum. Since an object of any kind resting on a surface will project 
above the surface, seeking a local maximum will direct the effector to a 
position above the object. 

For an object such as a sphere or even an irregularly shaped but 
convex object, quite accurate positioning will occur. For more irregular 
shapes, for example, a bowl, a more complicated control scheme could be 
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considered, but this configuration would loosely track the edge of the bowl, 
which projects upwards. 

A block diagram, of the described system is given in Fig. 5. 

Since the purpose of this experiment was to show feasibility of a 
concept, several simplifications were made. Actuation rates were kept low 
enough to ensure stability, motion was restricted to a vertically oriented 
plane, and the general joint- angle transformations were bypassed. 

A specific configuration of the manipulator, shown in Fig. 6, was used 
to avoid the necessity of performing the joint angle transformations within 
the control system. This technique may be applicable to simplifying other 
experiments where completely unrestricted motion of the manipulator is not 
required, and so it will be briefly described here. 

Oriented as shown, independent actuation of only one motor produces 
motion of the hand along one of the orthogonal x, y, z directions, in the work 
space. Vertical or z motion is by the shoulder lift joint, x motion is 
associated with shoulder rotation, and y motion with the elbow joint. If 
motion of the hand is restricted to a volume with linear dimensions small 
compared to the arm links, then the association of one motor with each 
orthogonal axis motion remains valid. The useful volume was of the order 
of a 30-cm cube. 

In our experiment the sensor -derived vertical control input was fed to 
the shoulder lift motor, and the horizontal input to the shoulder rotation 
motor. The elbow joint was not energized, and motion was confined to the 
x-z plane. Control of motion in a plane could be based on two sensors, 
while full three dimensional implementation would have required four 
sensors. 


IV. RESULTS 

The observed behavior of the sensor-controlled manipulator will be 
described here qualitatively, with the aid of the sketches in Fig. 7. 

A three minute movie entitled "Manipulator Control With Proximity 
Sensors" was made showing acquisition of several test objects by the sensor 
controlled effector. 
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On energizing sensor control, the hand would be driven toward a flat 
work surface, stopping in stable equilibrium with fingertips just above the 
surface, Fig, 7a, If an object resting on the surface is encountered by one 
of the sensors, lateral motion immediately results, ending with the finger- 
tips centered over the highest point of the object. Fig. 7b. The manipulator, 
of course, lifts if necessary to avoid contact as it centers itself over the 
object. With a rounded object of appropriate size for the Koelsch fingers, 
merely closing the fingers would result in grasping. The effector followed 
random motion of the object in the x-z plane, within the velocity capability 
of the manipulator (Fig, 7c). The observed accuracy of this positioning was 
approximately ±5 mm in both x and z directions. 

If a sloping surface is encountered, uphill motion will result, con- 
tinuing until a local high point is reached, Fig, 7d. 

A unique type of instability was observed with a roughly- cubic object 
having the length of one side approximately equal to the spacing between the 
two sensed volumes. A slight motion laterally moved the sensed volume on 
or off the object, and, as a result, limit cycle motion of 2- to 3- cm amplitude 
involving both x and z was observed. A number of techniques for eliminating 
this type of instability are available, but investigation of them was beyond 
the scope of this experiment. 


V. DISCUSSION 

This simple two-dimensional experiment illustrates how proximity 
sensors might be used to guide a manipulator hand during a grasping opera- 
tion. Four sensor s are neces sary for an analogous three-axis system. A pos- 
sible configuration is indicated in Fig. 8 where the sensors are shown 
attached to the fingers rather than the wrist area. The sensor-determined 
vertical position would be derived from a summation of all four outputs. 
Motion in the plane perpendicular to z would depend on two independent sig- 
nals obtained by subtracting the outputs of oppositely located sensors; for 
example, = Vj - V 3 , S y = V 2 - V 4 . 

Although beyond the scope of this discussion, other combinations could 
potentially be used to yield a control signal for finger closure, where 
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cl 


< V 1 + V 2> ' (V 3 + V 


and a wrist rotation signal S 

W 

S w = (V, + v 3 > - (V 2 + V 4 ) 

To use a similar technique in a real application, joint- angle transfor- 
mations in real time would be a necessary part of the control system. It 
would also be undesirable to restrict the effector orientation to the vertical 
alignment of Fig. 4. Allowing an arbitrary link configuration, and in addi- 
tion an arbitrary effector orientation, would mean that the sensor signals, 
obtained in an effector -fixed coordinate system, must be processed to 
derive the appropriate commands for each joint motor. It is this processing 
that has been referred to as the joint-angle transformation problem. 
Normally, a small digital computer has been used in real time within the 
control loop to perform the necessary computations, although other means 
could perhaps be developed. 

A second refinement desirable in a practical environment involves 
development of an effective technique for integrating sensor control into a 
system. The term '’reflective control" has been used to signify an approach 
in which the operator -manipulator loop is left functionally undisturbed at all 
times, but where operator commands can be modified or over-ridden by 
the sensors. Some means for superimposing the operator and sensor com- 
mands and properly weighting them is required. A satisfactory solution to 
this problem would greatly increase the effectiveness of the sensor- 
manipulator combination over that available with a simple ope rate r- 
controlled switchover between direct control and sensor control. 

Although the differencing and amplification of two separate sensor 
outputs might be expected to be subject to noise problems, difficulty was not 
experienced, because the signal was relatively strong. However, attention 
was given to controlling offsets and drifts in the subtractive channel that 
controlled horizontal motion. 

Another potential problem is that of variable reflectivity of the sensed 
objects themselves. Since the sensor output is based on the intensity of a 
light reflection, the function relating sensor output to position is dependent 
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on reflectivity. The objects used were of fairly uniform reflectivity. 
Although the system was not unduly sensitive to reflectivity; (for example, a 
granitic rock will serve as an object), extreme variations in reflectivity can 
be expected to cause errors in positioning. Two approaches can be sug- 
gested. The sensor response is partly determined by geometry and partly 
by the intensity of the light return. Optical modifications to increase the 
sharpness of the sensed volume could be made, resulting in an output tied 
more to the geometry built into the sensor optics. Alternatively, the sensor 
design could be changed to correct its response for reflectivity variations. 
Reflectivity variations such as one would encounter with a partially black 
(R a 5%) and partially white (R a 90%) object could be handled by one of these 
approaches . 

However, a highly reflective mirror surface represents an extreme 
case. Objects reflected in the mirror would be detected rather than the 
mirror surface itself. Diffuse scattering from a specular surface can be 
well below 1%, the result being too great an intensity range for the sensing 
device to properly account for. 
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Fig, 1. Replaceable external lens of lucite used on the proximity sensor 



Fig. 2. Internal configuration of proximity sensor showing knife edges 
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OUTPUT, volts 



Fig. 3. Observed sensor output as a function of object 


position. Null point of vertical control loop 

was set at V . 

o 



Fig. 4. Sketch showing the location of sensor pair on 
the Koelsch hand. 
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SENSOR 


MANUAL 



Fig. 5, Block diagram of the two -axis 
control scheme. 



Fig. 6. Configuration of the manipulator 
that was used to avoid the 
necessity of general joint angle 
tr ans f o r mation s . 
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Fig. 7. Examples of motion paths of the 
Koelsch hand under closed loop 
control. 


Fig. 8. Possible four sensor 
configuration for full 
three-axis control. 
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